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SUMMARY 


The  simplex  method  of  solution  of  a  linear  proj-ram  first 
transforms  the  original  system  to  an  equivalent  system  of  m 
equations  in  canonical  form  by  an  elimination  of  m  of  the  n 
unknowns.  If  the  right  choice  of  m  variables  is  made,  then  by 
equating  the  remaining  variables  to  zero,  an  optimal  solution 
is  obtained  to  the  original  problem.  If  not,  the  method 
produces  an  Improved  set  of  m  variables  and  a  corresponding 
canonical  form.  The  procedure  is  Iterated  until  an  optimum 
solution  is  obtained. 
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SIMPLEX  :iET.:QD 

algebra  of 

Introduction 

"simplex"  Is  a  mathematical  term  meaning  a  k-dlmenslonal 
"triangle";  l.e.,  It  is  the  generalization  of  tie  word  triangle 
for  two  dimensions  or  of  a  tetrahedron  for  three  dimensions. 

The  term  "simplex  method"  is  derived  from  one  of  several  geome¬ 
tric  interpretations  of  the  technique  in  which  this  figure  plays 
a  role.  In  this  lecture  its  mathematical  characteristics  will 
be  developed.  It  will  be  shown  that,  except  possibly  for  a 
member  of  a  class  of  cases  called  "degenerate,"  the  simplex 
algorithm  will  produce  In  a  finite  number  of  Iterations  a  solution 
(if  one  exists)  which,  minimizes  a  linear  form  in  non-negative 
variables  subject  to  a  system  of  linear  equations.  The  excepted 
class,  which  is  important  from  the  viewpoint  of  theory  (rather 
than  practice)  will  not  be  considered  here,  however  the  rule  of 
random  choice  given  for  resolving;  the  degenerate  case  can  be 
shown  to  lead  to  an  optimal  solution  In  a  finite  number  of  Iter¬ 
ations  with  probability  one. 

The  chief  feature  of  the  method  is  that  It  calls  for  elimi¬ 
nation  of  variables  from  the  equations  in  a  manner  quite  analo¬ 
gous  to  ordinary  (Gaussian)  elimination  for  solving  a  simultaneous 
system  involving  m  equations  In  m  unknowns.  After  an  elimination, 
using  some  selected  set  of  m  variables,  a  solution  is  obtained 
by  setting  all  remaining  variables  equal  to  zero.  If  this 
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solutlon  is  feaslr.le  it  can  be  used  as  a  starting  solution  for 
the  standard  simplex  algorithm.  The  next  step  is  a  test  to 
determine  whether  it  is  a  minimum  solution.  If  not,  a  new 
variable  is  chosen  to  be  eliminated  and  one  of  those  variables 
previously  selected  is  dropped  from  the  selected  set.  "^hla  will 
give  rise  to  a  new  feasille  solution  whose  cost  z  is  lower  than 
the  previous  solution  (in  the  ^ase  of  degeneracy  referred  to 
earlier,  it  may  have  the  same  cost).  Tie  procedure  may  be 
Iterated  and  it  can  be  o'-own  that  in  a  finite  nuinl  er  of  steps 
an  optimal  solutlot  can  he  ob  tained  (if  one  exists).  Since  it 
is  necessary  to  have  an  initial  solution  that  is  feasllle,  \'\e 
procedure  is  divided  into  two  p-.ases:  In  pi-iase  I  a  starting 
feasl'le  solution  (if  It  exists)  is  obtained  for  phase  II. 

The  Problem 

Find  values  of  ,  . . , ,  x,^  satisfying  the  simultaneous 

system  of  equations 


(1) 


+  a 


+ 


a,  X 
In  n 


'1 


^2n^n 


+  a 

m2  2 


+  a_ 


mn  n 


‘Vi 


and  the  Inequalities 
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which  minimize  the  linear  fo^-m 
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(?) 


CjXj  4-  CgXg  +  .  .  .  C.^X^  =  Z, 


where  j Cj  are  constarts  (i  =  1,  2,  ...,  in;  j  =  1,  2,  ...,  n) 
and  z  denotes  the  value  of  the  form  being  minimized. 


The  Canonical  Form 


Let  us  suppose  that  1  t  has  been  determined  that  a  starting 
feasible  solution  can  be  obtained  using  some  known  set  of  m  of 
the  n  variables.  By  relabeling  the  subscripts  the  selected 
variables  can  be  arranged  to  be  (x^,  x^^,  ...>  x^).  The  first 
step  Is  to  eliminate  from  the  cost  form  and  from  all  but  one 
of  the  equations  (l).  By  relabeling  the  subscripts  designating 
the  different  equations  the  equation  in  which  x^  has  not  been 
eliminated  can  be  arranged  to  be  the  first.  The  second  step  Is 
to 'try  to  eliminate  x^  from  the  modified  cost  form,  the  first 
equation,  and  all  but  one  of  the  modified  set  of  equations 
1  =  2,  ...»  m.  If  possible,  this  gives  rise  to  a  second  set  of 
modified  equations  and  a  cost  form  In  which  appears  only  in 
the  first  equation  and,  by  relabeling,  Xp  appears  only  In  the 
second  equation.  By  continuing  the  elimination  In  this  manner, 
if  possible,  for  variables  ...,  x  In  turn,  the  final  modified 
system  has  the "canonical”  form 


^  ^m+1  " 

*2nn-l 


4-  a^^x^  4-  ...  +  »  b^ 

4-  4-  ...  4-  -  bg 


^m4'l  !Tl4- 1 


+  X,  4-  ...  4- 


0 
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where  a^j,  Cj,  bj^  are  the  modified  values  of  a^^j,  Cj,  due  to 
the  eliminations  and  is  some  constant.  (Problem:  Prove  the 
values  of  are  independent  of  the  order  of  elimination.) 

In  certain  oases  It  is  not  possible  to  perform  the  elimi¬ 
nation  In  the  manner  prescribed  and  achieve  the  canonical  form. 
This  would  be  the  case  If  there  are: 

(a)  redundant  equations; 

(b)  Inconsistent  equations; 
and  may  be  the  case  for 

(c)  an  arbitrary  choice  of  the  set  of  m  variables. 

Since  methods  we  shall  discuss  for  solving  the  L.P.  problem 
depend  on  standard  methods  for  solving  a  system  of  linear  equa¬ 
tions,  an  appendix  has  been  added  to  the  end  of  this  Section 
for  review  ‘of  properties  of  such  systems. 

A  set  of  m  variables  is  called  a  ’’basic  set"  If  It  Is 
possible  to  use  them  to  reduce  the  system  of  equations  to  the 
canonical  form.  There  are  three  ways  usually  employed  for  deter¬ 
mining  whether  a  given  set  of  variables  Is  a  basic  set: 

(a)  Showing  the  determinant  of  the  coefficients  of  the 
selected  variables  Is  non-vanlshlngj 

(b)  Showing  a  set  of  values  of  the  selected  variables  can 
be  determined  (necessarily  unique)  whatever  be  the  values  of  the 
right-hand  side; 

(c)  Trying  to  carry  out  the  eliminations  In  the  manner 
prescribed. 

Problem:  Show  that  If  either  (a)  or  (b)  or  (c)  Is  true  the 
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other  two  are  true  aleo.  Prove  necessity  of  unlqjenesa  In  (b). 


The  Initial  Baste  Feasible  Solution  and  Test  for  Optimality 

The  standard  simplex  method  works  only  with  basic  solutions 
which  are  feasible.  By  a  "basic  solution"  Is  meant  a  solution 
of  the  constraint  equations  (l)  obtained  by  equating  all  non-baslc 
variables  to  zero.*  In  the  canonical  form  the  basic  solution  Is 
obvious — namely 


(5)  Xj  -  bj,  ■=  b^,  . . . , 

with  a  cost 


(6)  z  -  Zq  . 

’  it  is  a  basic  feasible  solution  If  the  values  of  the  basic  varlabl 


are  non-negative: 


(7)  >  0,  bj,  >  0,  b^  >  0  . 

In  canonical  form  It  is  also  easy  to  determine  whether  a 

basic  feasible  oclutlon  Is  optimal.  First  we  Introduce  some 

terminology.  We  shall  use  the  term  "cost  factor"  to  denote  the 

coefficients  c.  In  the  cost  or  objective  form  )'5)-  iN’e  shall  use 

the  term  "relative  cost  factor"  to  denote  the  coefflent  Cj  of  the 

variable  x,  In  the  cost  or  objective  form  of  the  canonical  system. 
J 

It  Is  relative  because  It  depends  on  the  choice  of  the  basic  set 


t. - _..j 

'^A  variant  of  this  procedure  can  be  developed  In  which 
non— basic  variables  are  equated  ho  cotistant  values  other  than 
zero. 
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of  variables.  In  particular,  the  relative  cost  factors  of  the 
basic  variables  are  always  zero. 

Theorem  1; 

A  basic  feasible  solution  la  a  minimal  feasible  solution 
with  total  cost  Zq  if  all  relative  cost  factors  are  non-negative: 

(8)  (j®l»2,...,n). 

Theorem  2: 

Given  a  minimal  basic  feasible  solution  with  relative  cost 
factors  “Cj  >  0,  then  any  other  feasible  solution  (not  necessarily 
basic)  with  the  property  that  x^,  =  0  for  all  c j  >  0  Is  also  a 
minimal  solution;  moreover  one  with  the  property  that  at  least 
one  for  some  Cj  >  0  can  not  be  a  minimal  solution. 

I 

Corollary; 

A  basic  feasible  solution  Is  the  unique  minimal  feasible 
solution  If  all  Cj  >  0  for  non-baslc  variables. 

Referring  to  the  canonical  form  It  Is  obvious  that  If  the  coef¬ 
ficients  of  the  cost  form  are  all  positive  or  zero,  the  smallest 
value  of  the  left-hand  side  Is  zero  for  non-negative  Xj.  Thus, 
the  smallest  value  of  z-z^  Is  zero  and  hence 

Min  z  =  Zq 

and  the  solution  is  optimal.  Problem:  Prove  the  other  theorems. 

Improving  a  Non-Optlmal  Basic  Feasible  .Solution 

On  the  other  hand,  If  at  least  one  relative  cost  la  negative. 
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Y 

It  l8  possible  to  construct  a  new  basic  feasible  solution  with 
a  total  cost  lower  than  z  =  z^.  (There  Is  exception  to  this 
statement  for  certain  "degenerate”  cases  that  we  shall  discuss 
later.)  Indeed  we  can  obtain  a  lower  cost  solution  by  Increasing 
the  value  of  one  of  the  non— l-iaslc  variables  and  adjusting  the 
values  of  the  basic  variables  accordln-7ly  where  x„  Is  any 
variable  whose  relative  cost  factor  Is  negative.  In 
particular  the  Index  s  can  be  cl osen  srch  that 


(9)  Cg  =  >iln  Cj  <  0  . 

The  latter  is  the  rule  for  choice  of  s  followed  In  practical 
computational  work  because  It  l.as  been  found  that  It  usually 
leads  to  fewer  Iterations  of  the  algorithm. 

Using  the  canonical  form,  let  us  construct  a  solution  In 
which  X  takes  on  some  positive  value,  the  values  of  all  other 
non-fcaslc  variables  are  still  zero,  and  the  values  of  the  basic 
variables  are  adjusted  to  take  care  of  the  Increase  In  x  : 

9 


(10) 


*1  ” 

*2  “  ^2  - 


m 


-  F  - 
m 


The  total  cost  associated  with  this  solution  Is 


(10. i; 


0 


+  c_x 


s  s 


It  is  clear,  since  c„  has  been  chosen  negative 
the  value  of  the  smaller  wlDl  be  the  value 


,  that  the 
of  z .  The 


larger 

only 
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thing  that  prevents  setting  x  Infinitely  large  Is  the  possibility 
that  the  value  of  one  of  the  basic  variables  In  (lO)  will  become 
negative.  Indeed  the  following  theorem  Is  clear. 

Theorem  3: 

If  for  some  a,  all  coefficients  of  x  In  the  canonical  system 

-  -  g  - i - 

are  negative  (or  zero),  Includln::  Its  relative  cost  factor,  then 
a  class  of  feasible  solutions  can  be  constructed  such  that 
z  — >  -00  as  x^  — >  +00  . 

On  the  other  hand,  If  at  least  one  Is  positive,  It  will 
not  be  possible  to  increase  the  value  of  x_  Indefinitely,  because 
beyond  the  ratio  b^/a^g,  the  value  of  will  be  negative.  If 
a^g  is  positive  for  several  1,  then  the  smallest  of  such  ratios, 
whose  subscript  will  be  denoted  by  1  =  r,  will  determine  the 
largest  value  of  x„  possible  such  that  all  values  of  x.  In  (lO) 
remain  non-negative.  Let  x*  *  max  x_  possible:  then 

8  S 


(11) 


♦  b  b. 

~  "  Min  >  0 

*1B 


where  r,  In  case  of  a  tie,  may  be  chosen  as  the  one  with  the 
smallest  Index  or  at  random  from  among  those  tied. 

Theorem  4: 

If  in  a  basic  feasible  solution  the  values  of  all  basic 
variables  are  strictly  positive  and  If  for  some  s,  the  relative 
cost  factor  Is  negative  and  at  least  one  other  coefficient 
©f  x„  In  the  canonical  system  is  positive,  then  a  new  basic 
solution  can  be  constructed  with  lower  total  cost  z. 
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Indeed  it  follows  from  the  assumption  that  all  >  0,  that 

* 

the  value  of  determined  by  (11)  Is  strictly  positive. 

o  5 

Conalder  the  feasible  solution 

*1  ”  ^1  - ‘is^  i  °  (1  =  1,  2 . m) 

(12)  >  0) 

Xj=0  (j=m+l,...,n) 

(J  /  b)- 

The  total  cost  of  such  a  solution  Is 
(IJ)  *  "  *0  *  <  *0- 

There  remains  only  to  show  that  such  a  solution  Is  a  new  basic 
feasible  solution.  It  Is  clear  from  the  definition  of  the  Index 
1  =  r  that 

(l4)  x*b— ax*"0 

and  that  we  are  tryln^^  to  show  that  x, ,  x_  (excludlnp 

Xj,)  and  Xjj  constitute  a  new  basic  eet  of  m  variables.  To  prove 
this  we  simply  observe  that  alnce  a^.^  >  0,  we  may  use  the  r-th 
equation  of  (4)  and  as  "pivot  element"  to  eliminate  the 
variable  Xg  from  the  other  equations  and  the  minimizing  form. 

It  Is  clear  that  we  have  again  reduced  our  system  to  canonical 
form. 

This  ability  to  obtain  a  new  canonical  form  from  the 
previous  one  by  means  of  a  single  additional  elimination  consti¬ 
tutes  the  key  to  the  computational  efficiency  of  the  simplex 


method. 
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Itsratlve  Proccduro 


The  new  basic  feasible  solution  can  now  be  tested  for  opti¬ 
mality.  If  not  optimal  then  one  may  choose  by  criterion  (9)  a 
new  variable  x.  to  Increase  and  proceed  to  construct  either: 

(a)  a  class  of  solutions  In  which  z  -<0  as  x  ->  i-co  (if 

9 

all  a^g  <  0)  or  (b)  a  new  basic  feasible  solution  In  which  ti.e 
cost  2  is  lower  than  the  previous  one  (providing  the  values  of 
the  baalc  varlablaa  for  the  latter  are  strictly  positive;  other¬ 
wise  the  new  value  of  a  may  be  equal  to  the  previous  value  of  z). 

The  simplex  algorithm  consists  of  iterating  the  above  "cycle" 
again  and  again,  terminating  only  when  there  has  been  constructed; 

(a)  a  elate  of  feaeible  solutions  in  which  z  -«d  ;  or 

(b)  an  optiaal  baaie  feasible  solution. 


If  on  each  eielt  tha  value  of  I  dfcraasec  a  non-sero  amount, 
then  the  entire  proeeet  will  terminate  in  a  finite  number  of 
cyclee.  The  reason  for  tMi  t»  there  are  only  a  finite  number 
of  ways  to  ehocst  a  tat  of  m  basic  vtriahlee  out  of  n  variables. 
If  the  algorithm  were  to  continue  indefinitely,  It  could  only 
do  00  by  repeating  the  same  basic  set  of  varlablss— hence  the 
same  cost  t. 


Degeneracy 

A  basic  solution  Is  said  to  be  "degenerate"  If  at  least 
one  of  the  values  of  the  basic  variables  is  zero.  In  this 
case  it  Is  clear  by  (ll)  that  If  for  some  >  0,  It  happens 
that  the  corresponding  value  of  the  basic  varlabl©  la  zero, 
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tiien  no  increase  In  Is  possible  (tiiat  will  maintain  the  values 
of  the  basic  variables  non-nei^atlve) .  It  follows  also  that  there 
is  no  decrease  in  z  for  that  iteration.  Tn  this  case  one  can  no 
longer  arirue  that  the  procedure  will  terminate  in  a  finite  number 
of  Iterations  because  with  no  chant, e  in  tl.e  value  of  z,  it  is 
conceivable  that  the  same  basic  set  of  variables  may  reoccur,  say, 
after  k  iterations.  Hence  if  one  were  to  continue  with  the  same 
selection  of  s  and  r  for  each  Iteration  as  before,  the  same  basic 
set  would  reoccur  after  2k  Iterations  and  again  after  3k  iterations 
etc.,  Indefinitely.  If  this  happens  it  is  referred  to  as  "cyclln,-" 
In  the  elaplei  algerlthii.  Theoretical  examples  of  eye  liny  have 
been  eonstructe^. 

The  frequency  of  oeeuffence  of  degeneracy  in  practice 
le  a  cariavi  phenomenon,  for  degeneracy  can  happen  only  when 
the  valaee  or  the  ortfinal  right-hand  tide  in  (l)  bear  a 
epecial  relation  to  the  tfoeffleiente  of  the  basic  variables. 

Thie  le  elear  einot  the  pfocece  of  reduction  to  canonical  form 
depend#  only  on  the  ooefftelente  and  not  on  the  right-hand  side; 

.the  final  ealuee  «re  eeighted  iunie  of  the  ori.ylnal  b^'s  where 
the  eelghte  depend  only  An  the  coefflciente.  If  the  b^^  were 
selected  at  random  it  ecoid  be  something  of  •  miracle  if  b^ 
should  vahtsh. 

One  way  to  place  the  entire  theory  on  a  rigorous  foundation, 
le  to  give  precise  rules  for  slightly  altering  the  values  of  the 
b^  so  as  to  avoid  degeneracy  and  thereby  the  possibility  of 
cycling.  However,  discussion  of  this  technique  will  be  post¬ 
poned  to  later.  In  practical  applications  the  phenomenon  of 
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cycllng  has  never  been  observed  In  spite  of  the  fact  that  dege¬ 
nerate  basic  solutions  ere  frequently  encountered.  For  this 
reason  It  Is  recommended  that  the  algorithm  be  applied  v>1thout 
any  special  rules  to  resolve  the  case  of  degeneracy. 


Appendix;  Syotema  of  Linear  Equations 


Equivalent  Systems;  Since  the  simplex  method  for  solving 
the  linear  programming  problem  depends  on  standard  methods  for 
solving  a  system  of  linear  equations  (which  of  course  Is  a 
special  cast),  we  shall  review  properties  of  such  systems. 

Suppose  that  we  have  a  syitem  of  m  equations  In  n  unknowns 
(l).  A  solution  of  the  l-th  equation  It  a  set  of  numberf) 

Xj,  euch  that 


(15) 


*11*1  ^  *12*2  ^ 


Ir  n 


A  solution  of  the  syeten  (l)  !«  a  set  of  numbers  which  Is  a 
solution  of  every  equation  of  the  system. 

Suppose  there  ti  anether  system  of  equations 


(16) 


*11*1  ^  *12*2  *  'ir^n  *' 


*21*1  *22*2 


^  •2n*n  -  ^2 


X. 


+  a, 


m2‘2 


^  X 

mji  n 


The  two  systems  (l)  and  (16)  are  call<=-d  equivalent  If  every 
solution  of  each  is  a  solution  of  tiie  other. 

The  process  of  solving  a  system  of  equations  is  thac  of 
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flndlng  an  equivalent  SjStem  of  simplest  form. 

Let  us  consider  system  (l).  Let  k^,  k^,  k^  be  any 

numbers  and  let 


(17)  *{1*1  *22*2  *in*n  "  ^1 

be  a  linear  equation  formed  by  a  linear  combination  of  equations 
from  (l),  l.e.,  formed  by  multiplying  the  first  equation  by  k^, 
the  second  by  k^,  etc.  and  summing  the  products,  'de  are  assumint’^ 


(Ifi) 


*il  “  *  ‘^2*21  ^  ^rn\l 

*ln  *  '‘l*ln  *  ♦  '  •  *  ‘'mV 


Consider  a  modified  System  of  equations  In  ehlch  all  equations 
are  the  same  as  (I)  eieept  the  l-th  equation  is  replaced  by  (17), 

i.e.. 


(19) 


*21*1  *f2*2  ^  ^  *£n*n  " 

*ml*l  *m2*2  ♦  •  •  *  +  •mn*n  " 


Clearly  every  solution  of  (l)  le  a  solution  of  system  (l9). 
Conversely  let  x^,  . . . ,  be  any  solution  of  (19).  It  is 
evidently  the  solution  of  every  equation  of  (l)  except  possibly 
the  i-th  equation.  But  the  substitution  of  (l8)  Into  the  2-th 
equation  of  (19)  reduce.^  to 


(20) 


k^(a^lXi 


+ 


^  2m*m 


0, 
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so  If  /  0  It  Is  also  true  that  the  i-th  equation  of  (l)  Is 
satisfied . 

Theorem: 

If  In  a  system  of  equations  (l)  the  £-th  equation  Is  replaced 
by  the  sum  of  equations  of  the  system  after  multiplication  by 
the  numbers  kj,  ...»  where  k,.  /  0,  then  the  new  syotem 
Ifl  equivalent  to  the  given  system. 

All  methods  of  solving  a  system  of  equations  employ  the  above 
principle. 

Elementary  Cperatlone;  There  are  two  kinds  of  elementary 
operatlone  which  can  be  performed  upon  the  equations  of  a 
system  to  yield  an  equivalent  system. 

Type  I:  Replace  anji  equatlof^ 

*11*1  ^  *12*?  ♦  ♦  a^^a^  - 

by 

where  k  /  0. 

Type  II:  Replace  any  equation 

^11^1  *12^2  ♦  ♦  *irj^n  “  ^1 

by 

^11^1  ^  ^  *ln*n  *  ^  ^  ^^n^n^  ^  ^ 

where  1  / 

Equivalent  Triangular  and  Canonical  m  x  m  oystems:  A  system 


of  m  equations  In  rn  unknowns  Is  trangular  If  It  has  the  form 


ml  1  m2  c  mm  m  m 

where  the  coefficients  along  the  diagonal  are  non-vanishing, 

(22)  a^^  ^0,  1  ■  1,  2,  ...,  m. 

It  Is  also  called  triangular  If  by  rearrangement  of  the  order 
of  the  equations,  and  relabeling  of  the  variables,  It  can  be 
written  In  this  form. 

Not  every  system  of  m  equations  In  m  unknowns  Is  equivalent 
to  a  triangular  system.  This  Is  the  case,  as  we  shall  see.  If 
the  original  system  contains  a  redundant  equation.  Also,  If  the 
original  system  has  no  solution,  l.e.,  Is  Inconsistent,  then  It 
could  not  be  equivalent  to  a  triangular  system,  since  solutions 
for  the  latter  can  be  readily  obtained.  For  example,  the  first 
equation  yields  the  value  x^  •  When  this  value  is  substi¬ 

tuted  Into  the  equations  1-2,  ...,  ra  the  values  of  the  remaining 
variables  can  be  obtained  by  solving  a  triangular  system  In  m  -  1 
equations  In  m  -  1  unknowns.  Accordingly,  the  problem  can  be 
solved  by  the  process  of  successive  substitutions.  This  process 
Is  really  the  same  as  a  sequence  of  elementary  operations,  l.e., 
multiplying  the  first  equation  by  adding  to  the 

i-th  equation  replaces  the  latter  by  an  equation  in  which  the 
variable  x^  Is  eliminated,  1  »  2,  ,  m.  In  a  similar  manner 
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Xg  may  be  eliminated  from  all  but  t'.e  second  equation.  11. Is 
yields  the  elmpleet  form  of  an  equivalent  system 


(23) 


=  b 


1 


-  b 


2 


X 


m 


This  Is  a  special  case  of  tie  "canonical”  form  for  linear  program¬ 
ming.  In  order  to  solve  a  system  of  m  equations  In  m  unknowns 
It  is  common  practice  to  try  first  to  "reduce"  It  to  an  equi¬ 
valent  triangular  system  bv  means  of  elementary  operations  and 
then  by  successive  substitutions  to  canonical  form  (25).  In 
linear  programming  type  work,  however,  It  Is  more  convenient  to 
raduce  It  directly  to  eanonleal  form  (If  possible). 

Reduction  of  m  bv  n  syatem  to  an  equivalent  canonical 
system;  A  s:)Stem  of  m  equations  in  n  (m  <  n)  unknowns  Is  In 
canonical  form  for  linear  programming  If  It  can  be  written 


(24) 


^m+l^mt 1 
31+1 


♦ 

+ 


m  m»m+l  mH 


^  ®1J*3 

+ 


nij  J 


+ 

+ 


In  n 


^  ^2n^i 


The  standard  procedure  for  reducing  (if  possible)  a  general 
system  (1)  of  m  equations  Ir  n  unknowns  to  equivalent  canonical 
form  will  now  be  discussed. 

Select  any  term  In  system  (l)  such  that  /  0.  Call 

this  the  pivot  element.  Rearrange  the  order  of  the  equations  and 
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the  variables  so  that  this  pivot  element  becomes  • 

The  elementary  operation  of  multiplying  the  first  equation  by 
~^11'^®11  adding  to  the  1-th  equation,  1  =  2,  3,  ...»  m, 

will  eliminate  Xj  from  the  remaining.  Multiply  the  first 
equation  by  l/a^^  • 

Let  a|j  denote  the  coefficients  In  the  new  system.  Con¬ 
sider  next  the  aystem  In  n-1  unknowns  x^,  x^,  ...»  x^^  consisting 
of  the  last  m-1  equations  and  repeat  the  process  of  selecting  a 
pivot  element,  rearranging  variables  and  equations  so  that  the 
pivot  element  Is  *22*2*  Multiplying  the  second  equation  by 
“■*12^*22  adding  to  the  1-th  equation  for  all  1  except  1=2 
produces  a  system  In  which  has  been  eliminated  from  all 
equations  but  the  second,  and  x,  from  all  but  the  first. 

Continue  In  this  manner  until  m  variables  are  selected 
for  the  elimination  process  or  until  after  selecting  r  variables 
it  Is  not  possible  to  find  a  pivot  element  among  the  remaining 
m — r  equations  and  n-r  variables.  At  this  stage  the  original 
system  has  been  reduced  to  the  e bivalent  system 

^1  ^  hr+l*r+l  »lrf2='r+2  *ln='n  "  h 

*2  *  *2r+l*rH  *  °2r+2’'rt2  ®2n’'n  "  *^2 
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where  the  variables  Ir  (25)  are  some  rearranj^ement  of  the  order 
of  the  variables  of  the  original  system  and  the  equations  In 
(25)  correspond  to  some  rearranc-ement  of  the  order  of  the 
original  equations.  The  number  r  Is  called  tie  rank  of  the 
system. 

The  original  system  has  no  solution,  i.e.,  Is  inconsistent 
unless 


(25.) 


b 


r?^ 


=■  0  . 


If  relations  (26}  hold,  this  means  that  each  of  the  m-r  equations 
of  the  original  S'-stem  correspondln:;:  to  equations  r+l,  . . . ,  m  of 
•the  reduced  svstem,  Is  redundant ,  I.e.,  Is  equal  to  a  linear 
combination  of  other  equations  of  the  system.  In  this  case 
one  can  solve  the  system  by  c^ooslnt:  any  values  for 
*r+2'  *n  then  detennlnlng  values  for  x^,  •••,  x^. 

Only  In  the  case  where  the  original  system  Is  neither 
Inconsistent  nor  redundant  can  the  system  be  reduced  to  equi¬ 
valent  canonical  form.  Here  the  rank  r  =  m.  The  set  of  m 
variables  In  the  original  system  corresponding  to  x^,  x^,  ..., 

X  Is  called  (as  noted  earlier)  a  ba.s1c  set  of  variables, 
m  ' 


II.  THE  SIMPLEX  TABLEAa;  AN  EXAMPLE 

The  "simplex  tableau"  Is  a  convenient  way  to  present  the 
canonical  system  In  detached  coefficient  form  In  whlcl".  the 
original  order  of  the  variables  and  the  equations  Is  kept  (instead 
of  rearranging  tnem  each  time).  A  simple  example  will  suffice 
to  Illustrate  tie  Idea. 
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Conelder  the  problem  of  minimizing  z  where 


(27) 


5Xj  -  ‘tXj  +  IJXj  -  2x^  +  Xj  *  20 

Xg  +  5X,  -  X,  +  Xj  "  8  (x,>0) 

Xj  +  6X2  -  7Xj  +  Xj^  +  5x^  =  z. 


J  - 


We  asBume  we  know  that  x^  and  x^  can  be  used  as  basic  variables 
and  the  baelc  solution  will  be  feasible.  Accordingly  we  proceed 
to  use  the  term  Xj  as  pivot  element  In  say,  the  second  equation 
(the  circled  element  of  (27))  to  eliminate  x^  from  the  other 
equations:  this  yields 


X2  -  12x^  f  ?Xj^ 


-20 


(28) 


-  Xg  +  5X^  -  Xj^  +  =  8 

Xg  -  12x^  +  2x2^  +  4x^  «  z-8  . 


Using  the  term  -4x^  as  pivot  element  In  the  first  equation  to 
eliminate  x^,  yields 


(29) 


—  ip  Xg  +  3x^  —  ^  Xji^  +  =  5 


X,  -  ^  Xr.  +  2X,  —  -  X) 


1  4  "2 


4  *4 


Sxg  -  24x^  5Xi| 


=  z  -  28  , 


which,  of  course.  Is  In  canonical  form  except  that  we  have  not 
bothered  to  rearrange  the  rows  and  columns.  The  corresponding 


tableau  is 
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(30) 


Ist  Basic  Solution  1 

i 

- 

!  Variables 

Variable 

Value 

i 

« 

Xg  Xj^ 

X.  ’  .  ' 

> 

1  ^ 

'  u  ©u 

-z  I  -28 

_  _  _i _ 

8  -24  5 

X  *  X 

where  the  coefflclentB  of  x^,  x^,  x^,  X|^,  x^  are  shown  In  detached 
coefficient  form.  Tne  basic  set  of  variables  (x^,  Xj.)  Is  recorded 
In  the  first  column  In  the  order  corresponding  ^o  the  position 
of  the  1  In  the  column  of  coefficients  of  and  x.^  respectively. 

The  symbol  -z  for  the  negative  of  tne  total  cost  Is  recorded  at 
the  bottom  of  this  column.  The  constant  terms  (which  are  also 
the  values  of  the  basic  solution  and  the  nep;at!ve  total  cost) 
are  recorded  In  the  "value”  column.  It  Ir?  helpful  when  iterating 
the  simplex  algorithm  to  place  check  or  x  marks  at  th;e  bottom  of 
the  columns  corresponding  to  the  variables  In  thie  basic  set  and 
a  star  In  the  column  with  the  most  negative  relative  cost  factor 
(in  this  case  'c^  -2^). 

It  is  also  convenient  to  place  a  circle  around  that  coefficient 

a  of  x  which  will  be  used  as  pivot  element  to  obtain  the  next 
rs  s 

Improved  basic  solution.  In  this  case  =  2  is  the  pivot  element 


because  ti'.c  smailest  oT  tl.e  positive  ratios  fcrnieQ  b.v  divic'ii.- 

the  numbers  b,  in  the  "value”  column  bv  tte  coefriclents  n, 

1  ”  is 

that  appear  In  the  "x  ”  column  occurs  for  the  ratio 

o 

X*  n  ^2/^23  *  3A  ^  smallest  oV  the  ratios  5/3, 

3/2).  This  means  that  x,.  will  replace  the  basic  variable  t;.at 

✓ 

appears  on  the  second  row,  x^,  In  t:  e  next  cycle  as  a  !;ew  basic 
variable.  Indeed,  eliminating  x^  from  all  but  the  second  equati 
the  next  tableau  becomes 

(51) 


It  will  be  noted  that  the  relative  cost  factor  for  Is  nega¬ 
tive.  The  basic  solution  Is  x,  =  3/2,  =  1/2  and 

The  total  cost  Is  given  by  -z  =  3  or  z  -  -8.  Since  the  solution 
Is  not  optimal  we  again  seek  an  Improved  basic  solution.  The 
smallest  positive  ratio  occurs  tlils  time  on  rov^  Ij,  hence,  7/8 
becomes  the  new  pivot  element.  Thus  in  the  next  tableau  Xp 
replaces  x^  as  a  basic  variable. 
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(??) 


^rd  Basic  Solution 

Variable 

Variable 

Value 

*2 

*3 

*5 

*2 

k/7 

-12/7 

1 

-V7 

8/7 

12/7 

-  1/7 

1 

-2/7 

3/7 

-z 

(.0/7 

72/7 

11/7 

8/7 

X 

X 

Since  all  relative  co::t  factors  are  nonnegative,  -.ne  solution 
shown  Is  minimal.  The  total  cost  Is  z  -  -fO/7  , 


III.  FINDING  AN  INITIAL  BASIC  FEASIBLE  SOLJTION 


Up  to  the  present  we  have  been  assuming  that  a  basic  set 
of  variables  could  be  specified  wnicf  could  be  used  to  perfoinD 
tie  necessar^  Initial  eliminations  and  reduce  tne  problem  to 
canonical  form.  It  has  also  been  assumed  that  the  associated 
basic  .solution  is  feasible. 

It  is  true  that  man;,  problems  encountered  in  practice  often 
have  a  starting  solution  readily  at  hand.  For  example,  for  the 
Important  class  called  "transportation"  problems  one  can  con¬ 
struct  immediately  a  great  variety  of  starting  basic  solutlons- 
Economlc  models  often  contain  storage  and  slack  activities  which 
permit  an  obvious  startln.g  solution  in  which  nothing  but  these 


t  '0!. 


1: d 


i-.e:'  fr^r 


ucl'lvltlcs  lo:«  . 


the  optlrnuiT,  colutlcr.  tut  at  loa.‘:t  \t  lb  an  ••.-acy  start,  tnually, 
little  effort  Is  required  in  those  cnees  to  rech.’co  the  problem 
to  canonical  form. 


On  ti'Q  ot'er  I'.and  many  prohl  n.r  cncom-'t'' rod  In  prantico 
do  net  provide  any  oby1o''s  start'ri’-  haelc  set  of  varlaM‘'’s.  In 
fact,  little  or  nothin,"  may  to  knoxn  (mal.-herriatlool  1,;  -peaking) 
about  t!ie  prc.blein  requiring  solution: 

(a)  It  .may  have  redundanclrc  —  for  example,  the  equation 
balancing  money  flow  may  have  been  obta'r,e-j  from  the  equations 
balancing  material  flows  by  multlpl'.  Ing  price  by  quantity  and 
summing;. 

(b)  It  may  have  Inconsistencies  due  to  outright  clerical 
errors  or  to  use  of  Inconsistent  data. 

(c)  It  may  'nave  Impossible  requirements  considering  the 
resources  available.  Thius  it  may  be  a  problem  In  which  resources 
are  known  to  be  In  short  supply  and  the  main  question  asked  is 
really  whetiier  a  feasible  solution  e.tls*'s  at  all. 

Prom  the  above  considerations  It  Is  ''•Tear  that  a  general 
mathematical  technique  must  be  developed  zo  solve  linear  pro¬ 
gramming  problems  free  of  an;.,  prior  knowledge  or  .e.-?sumpti  ons 
about  the  e.stems  being  solved.  In  fact,  if  there  are  Incon¬ 
sistencies  or  redundancies  these  are  important  facts  to  be 
discovered . 

A  simple  device  will  be  presented  In  this  section  whlc.’-’ 
uses  the  .simplex  algorithm  1t.self  to  provide  (if  It  exists)  a 


P-8.91 

7-9-56 

-24- 


starting  basic  feasible  solution.  Ihls  part  of  t!^.e  process  Is 
usually  referred  to  as  phase  I.  The  second  part  of  the  process, 
obtaining  an  optimal  basic  feasible  solution,  Is  then  referred 
to  as  phase  II.  The  device  has  several  Important  features  that 
should  be  noted: 

(a)  No  assumptions  are  made  regarding  the  original  system; 

It  ma;,  be  redundant,  Inconsistent,  or  not  solvable  In  non-negative 
numbers . 

(b)  ho  eliminations  are  required  to  obtain  an  Initial  solu¬ 
tion  In  canonical  form  for  phase  I. 

(c)  The  end  product  of  phase  lisa  basic  feasible  solution 
(if  It  exists)  In  canonical  form  read,'  to  initiate  phase  II. 


The  procedure  for  phase  1  Is  as  follows: 

.Step  1:  Arrange  tie  original  system  of  equations  so  that 
all  constant  terms  are  positive  (or  zero)  by  changing,  where 

necessary,  the  signs  on  both  sides  of  any  of  tliC  equations. 

Step  11:  Augment  the  system  to  Include  a  basic  set  of 
"artificial"  variables  >  0,  x^^^  >  ^n+m  - 


(3J) 


SjlXj  +  +  . 


4-  4  X„^1 

In  n  n4i 


ar\iX,~,  4  ano^n  . . .  "f"  a/.,,.x^ 


‘21 


‘22  2 


2n  n 


‘n42 


=  bj  (b^  >  0 

=  b„ 


and 

(3^) 


a  .X,  4  a  4-  ...  fa 
ml  1  m2  2  rnn  n 


4  X  ,  „  =  b 
n+m  m 


(j  =  1 , 2, . . . ,n;n+l , . . . ,n+m) 


Step  III:  Using  the  simplex  algorithm,  find  a  solution  to 
(35)  and  (34)  which  minimizes  the  sum  of  the  artificial  variables, 


denoted  bv  w. 


We  call  t'ils  the  "infeaslhlll ty  form" 
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(55) 


nf  1 


+  X 


n+2 


f  X 


n+m 


Step  rV:  If  Min  w  >  C,  t:\en  no  feasible  solution  exists 
and  the  procedure  is  terminated.  On  the  otfier  hand,  if  Min  w  «  o, 
then  all  =  0  and  t‘'o  basic  feasible  solution  constitutes 

a  feasible  solution  to  the  original  unaugmented  system.  Initiate 
phase  II  of  the  simplex  algorithm  by 

(a)  dropping  from  further  consideration  all  non-basic 
variables  whose  relative  cost  factors  for  form  w  are  strictly 
positive . 

(b)  replacing  the  linear  form  w  (as  modified  bv  various 
eliminations)  by  the  linear  form  z,  after  first  eliminatln.;  from 
2  all  basic  variables.  .It  is  common  computational  practice  to 
perform  the  elimination  of  the  basic  variables  from  z  on  eaci' 
cycle  of  phase  I.  In  this  case,  the  modified  i'orrn  z  may  be 
used  immediately  to  initiate  phase  11. 

Discussion:  While  the  procedure  for  phase  I  is  almost  self- 
evident,  it  does  deserve  some  diycussion.  j.t  is  clear  tliat  if 
there  exists  a  feasible  solution  to  the  original  system  (l)  and 
(2),  then  this  same  solution  also  satisfies  (3^)  and  (3^  with 
the  artificial  v.arlables  set  equal  to  zero.  w=  C 

this  case.  From  (35),  the  smallest  possible  value  for  w  (since 
it  is  the  sum  of  non-negative  variables)  is  zero.  Hence,  if 
feasible  solutions  exist,  the  minimum  value  of  w  will  be  w  ~  0; 
conversely,  if  a  solution  is  obtained  for  (33)  ^nd  (3^)  with 
w  ==  0,  it  is  clear  that  all  ==  0  and  the  values  of  x.. 
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for  j  <  n  constitute  a  feasible  solution  to  (l).  It  also  follows 
that  if  Min  w  >  0,  then  no  feasible  solutions  to  (l)  exist. 

It  should  be  noted  tliat  v/henever  the  original  system  contains 
redundancies  (and  often  when  degenerate  solutions  occur),  arti¬ 
ficial  varlab  1  es  v;lll  appear  as  part  of  the  basic  set  of  variables 
In  phase  II.  If  they  do  appear  It  Is  necessary  that  their  values 
never  exceed  zero.  This  Is  accomplished  In  Step  IVa  where  all 
non-baalc  variables  are  dropped  whose  relative  cost  factors  for 

\'i  (denoted  by  ^.)  are  positive.  The  justification  for  this 

J 

procedure  Is  based  on  theorems  1  and  2.  For  clarity  we  shall 
.  repeat  the  argument.  The  basic  variables  at  completion  of  phase  I 
will  appear  In  the  reduced  form  w  with  zero  coefficients;  In 
fact,  the  form  of  w  after  elimination  of  the  basic  variables 
will  be 

(56)  +  Vj  . . .  =  «  -  «o  i  "0  " 

where  =  0  for  basic  variables  and  all  H’j  >  0  because  the 
solution  Is  optimal  (theorem  l);  also  w^  >=  Min  w  «  0  since 
feasible  solutions  exist.  All  feasible  solutions  to  (l)  are 
feasible  solutions  to  (35)  with  =»  0  and  w  =  0.  They 
cannot  contain  an  Xj  >  0  with  >  0  because  this  implies  w  >  0; 
hence,  all  such  may  be  dropped  from  further  consideration. 

If  we  drop  them,  then  we  confine  our  attention,  as  In  phase  II, 
only  to  variables  whose  corresponding  3"^  «  0.  By  (56)  solutions 
Involving  only  these  variables  have  w  “  0,  and  consequently  are 
feasible  for  the  original  problem.  Thus 


Theore:.'!  ^ 


■If  r.rtlflclal  variables  for.-n  part  of  Ve  baolc  ^eUa  oT 
variables  Ir.  the  various  cycles  of  phase  II,  tlelr  values  will 
never  exceed  '/ero . 

To  illuetrate,  let  us  return  atraln  to  example  (27),  Since 
th<=.  constant  terms  are  all  positive  we  aup:ment  the  s;. stem  with 
the  auxiliary  variables  x,  and  and  consider  for  phase  1 

5x^  -  +  lyx,  -  2x^  +  -  2u  (x^  >  C) 

(p7)  Xj  -  x^  +  5x^  -  x,j  +  x^  +  X7  -  S 

x^  +  x,^  =  w.  (w  =  Min) 

Ellmlnatlnc;  tt.e  bas^c  variables  x^:  and  from  w  b;.  addin;:  the 
first  two  equations  and  subtracting  from  w  yields  the  starting 
tableau  1—0  (below)  for  phase  I.  .Applying  iteratlvel  '  t.he 
simplex  algorithm  yields  tableaux  1-1,  1—2,  and  II-O  which  Is 
tie  start  of  phase  11. 


Basle  Solution 

!  Variables 

Symbol  Value 

x^  X^  i 

1 

! 

2C 

..A 

13 

-2 

+  1 

1  : 

1 

Xy 

8 

1 

‘ 

-1 

'+5 

-1 

-n 

1 

— w 

-28  ii  -6 

-18 

+3 

“2 

1 

1 

r 

if- 

X :  X 

P-a9l 

7-9-56 

-28- 


1.5^ 

X’j  •  50 

.39 

-.92 

-.31 

+  .54 

1  1  -.15  !  .08 

-.23  +.62 

.08 

-.39 

1 

-w  ■  .-.31 

+.92 

L_ _ 

+.23  -.fi 

1.39 

1 

L  ix!  1  ^ 

_ 

■ 

X 

Iptimal ) 


On  the  first  Iteration  the  value  of  w  was  reduced  from  28 
to  on  the  next  Iteration  w  =  0  and  a  feasible  basic  solution 

=  1.5,  =  .5  Is  obtained  for  the  original  unaugmented  system. 

Variables  x^  and  x^  have  positive  relative  cost  factors  for  w 
and  hence  must  be  dropped  for  phase  IT.  Tableau  II-O  is  obtained 
from  1—2  by  dropping  columns  associated  with  these  variables,  and 
replacing  the  w  row  by  a  z  row  using  the  first  two  rows  of  1—2 
to  eliminate  x,  and  Xr-  from  z.  Because  of  the  canonical  form  of 


P-891 

7-^56 

-29~ 


1-2  this  elimination  Is  accomplished  by  multiply,  Imr  the  first 
and  second  rows  by  the  coefficients  of  x,  and  respectively 
in  form  z  and  subtracting  from  z.  By  food  luck  all  relative 
cost  factors  of  form  z  are  positive  in  li-O;  the  starting  fea¬ 
sible  solution  for  phase  IT  turns  out  also  to  be  the  optimal 
solution  (a  happy  accident)  and  therefore  no  further  iterations 
are  required. 


